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Abstract 

We treat observable operator models (OOM) and their non-commutative generalisation, which we call 
NC-OOMs. A natural characteristic of a stochastic process in the context of classical OOM theory is the 
process dimension. We investigate its properties within the more general formulation, which allows to 
consider process dimension as a measure of complexity of non-commutative processes: We prove lower 
semi-continuity, and derive an ergodic decomposition formula. Further, we obtain results on the close 
relationship between the canonical OOM and the concept of causal states which underlies the definition 
of statistical complexity. In particular, the topological statistical complexity, i.e. the logarithm of the 
number of causal states, turns out to be an upper bound to the logarithm of process dimension. 
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1 Introduction 



< 

' The main idea behind various complexity measures, such as statistical complexity, is the same that gave rise to 
the famous Kolmogorov complexity. Namely, the complexity is the "size" of some minimal "representation" 
of the object of interest. Different complexity measures are based on different exact definitions of these 
terms. For Kolmogorov complexity, for instance, representations are Turing machine programs computing 
individual binary strings, and the size is their length. For statistical complexity ([3]), on the contrary, the 
objects of interest are probability distributions of stochastic processes instead of individual strings, and the 
1^ ' representations are particular kinds of predictive models in the sense of partially deterministic hidden Markov 
, models (HMM). Their size is measured by the Shannon entropy of the internal states of the model. 

Observable operator models (OOM) are generative algebraic models that represent a stochastic process. 
The natural measure of size of an OOM is the dimension of the corresponding real vector space. It is minimal 
for canonical OOMs of a given stochastic process and was already identified as a characteristic of the process 
called process dimension. In the present contribution, following the above mentioned reasoning, we want to 
' consider the process dimension as a complexity measure for stochastic processes. We give further indication 
00 . that this might be appropriate. First, we show the close relation of the canonical OOMs to the concept of 
C^^ ' causal states which are used to define statistical complexity. Second, we prove that the process dimension, 
considered as function of the process, is lower semi-continuous. Although there exists no generally accepted 
00 ' axiomatic characterisation of functionals on the space of stochastic processes that quantify complexity, we 
\ argue that every complexity measure should feature this property. Indeed, it would be strange to consider 
a process complex if there is an approximating sequence with (uniformly) simple processes. The natural 
topology for processes is in this context weak-* topology as opposed to the much stronger variational topology, 
and lower semi-continuity w.r.t. weak-* topology is a much stronger result. 

The construction of causal states relies heavily on conditional probabilities. This makes it difficult to 
}J] ' extend the corresponding notion of statistical complexity to the domain of non-commutative processes un- 
5^ \ derstood to be states on a quasi-local C*-algebra. The algebraic formulation of OOMs, however, allows to 
extend the concept of process dimension to the non-commutative setting. Indeed, the construction oi finitely 
correlated states introduced by Fannes et al. in [4] provides OOMs for a class of shift-invariant states on 
a quasi-local C*-algebra. In the literature, these states are also known as algebraic states. In this paper, 
however, we refer to their original name. We show lower semi-continuity of the process dimension also in this 
more general setting. 

The outline of our paper is as follows. In Section 2 we present main concepts in the context of classical 
OOM theory using the terminology introduced by Herbert Jaeger. In particular, process dimension is defined. 
In Section 3 we review the concept of statistical complexity and the underlying notion of causal states which 
are defined for classical stationary process. In Proposition 6 we specify the way in which process dimension and 
causal states are related. As direct implication we obtain an upper bound for, and an ergodic decomposition of 
the process dimension in the classical case (Corollaries 7 and 8). In Section 4 and 5 we treat non-commutative 
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extensions. We start with the definition of NC-OOMs referring to finitely correlated states. As in the classical 
special case the corresponding process dimension is naturally associated with canonical NC-OOMs. Our main 
results are contained in Section 5. There we prove lower-semicontinuity and provide an ergodic decomposition 
formula for the process dimension in the general case. 

2 Classical OOMs (Stochastic Modules) 

Fix a. finite set A. We consider A-valued stochastic processes Xjt^ := {Xk)ke¥i, described by their distributions 
P € 'P{A^), and stationary processes Xz, described by their shift-invariant distributions P G Ps(A^). 

In [6], Alex Heller introduced a generalisation of functions of Markov chains, called stochastic modules. 
Later, Herbert Jaeger extended and reformulated this theory in the language of linear algebra ([7]). We use 
his terminology. 

Definition 1. An observable operator model (OOM) with alphabet A is a quadruple {V,T,v,£), where 
F is a real vector space, T: A x V ^ V is linear in the second argument, v is an element of V, and £ is & 
linear form on V, such that for Td{v) := T{d, v), n Gl^ and rfi, . . . , d„ e A, 

1. ^(t;) = 1, 2.£o^Td = £, Pd,,...4^ := ioTd„o...oTd,{v) >Q. 

The vector v is called initial vector, the operators Td are called observable operators and the linear form 
£ is called evaluation form^. The process P £ P{A^), defined by P{[di, ■ ■ ■ ,d„]) := Pdi,...,d„ {n € IN), is 
called generated by the OOM and the dimension dim(y) of V is called dimension of the OOM. 

It is easy to check that the Pdi,...,d^ are a consistent set of finite-dimensional probabilities. Therefore, by 
the Kolmogorov extension theorem, the process P well-defined. Every hidden Markov model (HMM) with n 
internal states canonically induces an n-dimensional OOM. For more details, see [7]. 

Remark. More generally, an HMM with set F of internal states can be interpreted as OOM with vector 
space M{r) of signed measures of bounded variation on F. 

There is a canonical construction of an OOM of a given process P G 7^(A'^). Let A^(A'^) be the space 
of signed measures of bounded variation on A**, i.e. A^(A''^) = span('P(A''*)) , where span denotes the linear 
hull. Define the linear maps : M{A^) MiA^) by 

r^{„) := „{[d]na-\.)) 

where a is the left-shift on A". Further define £a : M{A^) ^ R by £ a(m) = fJ-iA^), i-e. the evaluation form 
£a associates to a measure its total mass. For convenience we define 

AAA 

Definition 2. For P G P(A"^), let 

Qp {Tdi-dniP) I nGlNo, di,...,rf„ G A} and Vp := span(Qp). 

For d G A, denote the function Vp — > Vp, i-> r^ifJ.) with a slight abuse of notation again by t^. Set 
T^{d,(i) := T^{n). Then {Vp,t^,P,£a) is called canonical OOM of P. 

Since rfiVp) C Vp, the canonical OOM is well-defined, and it generates P. It has minimal dimension 
among all OOMs generating P, and is, up to isomorphism, unique with this minimality (see [7]). In particular, 
the dimension of Vp is not bigger (but may be essentially smaller) than the minimal number of internal states 
required for any HMM generating P. Another characterisation of Vp is in terms of conditional probabilities: 

Vp = span{p(a-"(-) | [di,...,rf„]) | n G INq, di, • • • , rf„ € A, P([di, . . . , rf„]) >o}. 

This is true because if we normalise pointwise, we obtain the corresponding conditional probability. Note 
that the normalised version of is not linear. 

If A is a finite dimensional cylinder set, the same holds for [d]nc7~^(A). Therefore, is weak-* continuous 
and, consequently, maps the weak-* closure Vp^* to itself. Thus (V^™* , r^, P, ^a) is an OOM of P, which 
we call the closed canonical OOM. In the case of finite process dimension, which we are mostly interested 
in, the canonical OOM and the closed canonical OOM coincide. For considering infinite pasts in the following 
section, however, the weak-* closure of Vp in M{A^) plays a crucial role. 



^ Jaeger fixes a basis of V instead of an evaluation form and defines i to be the sum of coefficients in the basis expansion. 
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Example 3 (Canonical OOM and Shift HMM). The (onc-sidcd) shift HMM of P e V{A^) is a deterministic 
HMM with set F := A''^ of internal states. It is in general by no means minimal and it is not possible to 
restrict it to a smaller subset of F such that it still generates P. If we interpret it as OOM, the internal 
Vector space is V = A4{A^), the associated operators Td arc equal to the canonical ones, i.e. Td = t^, the 
initial vector is the initial distribution of the shift HMM, i.e. v = P and the evaluation form is i-> ^(A'^). 
Now it is obvious that we can reduce every OOM to a "cyclic" version by restricting V to span{ T(ij...d^(i;) | 
n G Mo, di, . . . , d„ G A }. This reduced shift OOM is just the canonical OOM and thus minimal, but it can 
in general not be interpreted as HMM. 

Definition 4. The process dimension of P e V{A^) is the dimension of its canonical OOM (Vp, r^, P, £a)- 

dim(P) := dim(Vp) e INU{oo}. 

The process dimension is derived from a canonical construction and at the same time the minimal di- 
mension necessary for an OOM-rcprcscntation. Therefore, it is an important internal characteristic of the 
process and might be considered a complexity measure. As we see in the following section, it is related to 
statistical complexity. 

3 Infinite Pasts and Causal States 

Now assume we arc interested in a stationary process with infinite past, i.e. X^, described by its shift-invariant 
distribution P G Ps(A^). For simplicity assume that Xk is the canonical projection on A^. We define the 
canonical OOM of P to be the canonical OOM of its restriction to positive times, i.e. if Pjn := P o X^^ is 
the distribution of X^ then 

dim(P) := dim(P]N) = dim(VF^). 

In [3] , the causal states of such a stationary process were introduced and used to define statistical complexity. 
Causal states are equivalence classes of past trajectories, where two of them arc identified if they induce 
the same conditional probability distribution on the future X^. In this paper we prefer the alternative 
definition of causal states, where they are measures on the future, i.e. elements of 7'(A''^). This viewpoint 
was introduced in [8]. 

If we observe the past of X^, the observation = induces a certain conditional probability 

distribution P{Xfi \ = X-k^) G V{A^) on the future of the process. The causal state distribution 

of P G PsiA^) is the distribution of these conditional probabilities. In particular, it is a measure on measures. 
More precisely we define 

Definition 5. Let P G Ps(A^). The causal state distribution fi^ G P{P{A^)) of P is defined by 

:= Po{P{X^\X_^,))-\ 
where we consider P(X]n | -'^-ino) to be a (measurable) function from A^ to 7'(A''^). Further, we define 

Cp := supp(/z?) C V{A^). 

Remark, a) /i^ is the distribution of the P(A'^)-valued random variable P{Xn \ X_]n„). 

b) The causal states correspond to the elements in the image of P{Xf^ \ Thus, the set of causal states 
depends on the version of conditional probability. £p, on the other hand, is independent of the choice of 
conditional probability. 

c) The statistical complexity Ce;(P) := H{^^) is the (Shannon) entropy of the causal state distribution. It 
was originally introduced by Grassberger as true measure complexity in [5]. 

We obtain the following relation between the causal state distribution and the closed canonical OOM 
vector space. The weak-* closure of the canonical OOM vector space is equal to the weak-* closure of the 
vector space spanned by the support €.p of the causal state distribution. In the finite dimensional case, this 
means that the two vector spaces are eqiial. Because the OOM vector space is defined with finite-length 
pasts and infinite pasts are used for the definition of the causal state distribution, we can interpret this result 
as follows. Unlike the set of causal states, the canonical OOM vector space is the same if we consider finite 
or infinite pasts, provided it is finite dimensional. For the infinite dimensional case, the situation is more 
subtle (see Example 9). Note that V^* denotes the closure of V w.r.t. the weak-* topology. Recall that 
Vp = span{Qp). 
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Proposition 6. Let P G Ps(A^). Then 



Vp^* = span(£p)"'*. 

In particular, because finite-dimensional spaces are closed, dim(P) = dim(span(2;p)) . 

Proof. "C": Let y € Qp. Then u = r^...d„(P]N) for some c?i , • • • 5 dfji G A- Define A to be th.e event that the 
past is di, . . . , dn, i.e. A := { Xi-n+k = dk, fc = 1, . . . , n } C A'^. We assume P{A) > 0, as otherwise v = 0. 
Further define the non- normalised measure P := P{Ar\ ■ ) G ^4+{A'^) and denote = Po(P(X]n | X_]No))~^. 
Note that the conditional probability P{Xk \ X-^o) in the definition of fj, is w.r.t. to P not P. Using 
stationarity of P we obtain 

y = P]N([rfi,...,d„]na-"(-)) = yp(^n{X]NG • } I X_]No) dP = ^P(X]N |X_Ko)dP 

= j P(X]N I X_]nJ dP = j idp(AH) d/x = • 

where id is the identity, ||/x|| = IJ,(V{A^)) is the norm of total variation and r: V('P{A^)) — > P(A*') is 
the resultant (also called baryccntrc map) from integral representation theory. Because P P, and thus 
jjL <C jU^, the support of jj, is contained in £p. Due to compactness of £p, this implies that the bary centre 
lies in the closed convex hull of £p ([2]), i.e. 



At^) e conv(£p)"'* and v G span(£p)^ 



MIMII 

"D We have to show that Vp^* has full /i^-measure, in other words that P{Xk \ X-^^) G Vp""* P-a.s. 
By the martingale convergence theorem we have for all B G 5B(A™) a.s.: 



P({X]N G P} I X_^,){uj) = lim P({X,N G P} I X[_„,o])M = lim 



n— J-oo 



p([XiH,...,x„H]) 



Because *B(A ) is countably generated and setwise (pointwise) convergence of a sequence of probability 
measures implies weak-* convergence, wc obtain P{X-^ \ X^^^) G K, • Qp"'* P-a.s. □ 

Corollary 7. The logarithm of the process dimension is upper bounded by the topological statistical complexity 
(the logarithm of the number of causal states). 

As a second corollary, we obtain an ergodic decomposition formula for process dimension. Namely, the 
dimension of a process is the sum of the dimensions of its ergodic components. This is not too surprising, 
because ergodic measures are mutually singular. We prove this formula more generally in the not necessarily 
commutative case in Section 5. Nevertheless, we give an alternative proof for the classical case here. 

Corollary 8. Let P G Ps(A^) with ergodic decomposition v G P(Pe(A^)). Then 

dim(P) = dim(/x), 

where we use the convention that sums over uncountably many strictly positive elements are infinite. 

Proof. We use that dim(P) = dim(span(Cp)) by Proposition 6. It is evident that dim(P) cannot exceed the 
sum. Let Pi, . . . , P„ G supp(i^) be distinct ergodic components of P. Then there are disjoint Ai, . . . , An G 
Q5(A''^) s.t. Pfc({X]N G Ak }) = 1. Consequently, Pfc({X]N G Afc } | X^^o) = 1 Pfe-a.s. and, because Mfc = 
{ /i G P(A'^) I /i(Afc) = 1 } is closed. £p^ C Mk- The vector spaces span(Mfc) are obviously linearly 
independent and thus the vector spaces Vk — span(£p^) are linearly independent as well. As span(£p) D 
U/c ^k, we obtain dim(P) > dim(Pfe). □ 

Remark. Assume that £p is countable and all elements have non-zero /it^-probability, so that we can identify 

£p with the set of causal states. The e-machine of computational mechanics is an HMM with the set of causal 
states as internal states. The vector space corresponding to this HMM (if we interpret it as OOM) is M.{tp) 
as opposed to the canonical OOM vector space Vp = span(£p). The latter can be much lower dimensional, 
because it utilises the linear structure of £p. 
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The closures in Proposition 6 arc really necessary, as we see in the next example. Although €p is closed, 
span(£p) is not (in general). Also, in general, neither does span(Cp) contain Vp nor the other way round. 

Example 9. Let A = { 0, 1 } and for p S [0, 1] let Pp £ Ps(A^) be the Bernoulli process with parameter p, 
i.e. Pp is i.i.d. with Pp({ Xi — 1}) = p. Consider the uncountable mixture P = J Pp dp, where integration 
is w.r.t. Lebesgue measure. Then /x^ is the image of Lebesgue measure under the; map p i-^ Pp o X^^, and 
€p = { Pp o X^^ I p e [0, 1] } is the set of i.i.d. processes. We make the following observations: 

1. span(£p) n V{A^) is the set of finite mixtures of i.i.d. processes, in particular span((ip) is not closed. 

2. Vp has countable algebraic dimension, i.e. it is the linear hull of a coimtable set, while a basis of span(£p) 
has to be uncountable (the family {Pp)p£[o.i] is linearly independent). Thus, Vp cannot contain span(£p). 

3. All elements of Vp n 7-'(A'^) have an uncountable number of ergodic components. Therefore, span(£p) 
and Vp are even disjoint. 

4 Non-Commutative OOMs 

Since OOMs are, unlike the concept of causal states, formulated algebraically, they have a rather natural 
generalisation to the setting of non-commutative algebras: Intuitively, we have to replace symbols from 
an alphabet A by operators representing observables. More precisely, we pass from the algebra C(A) of 
(continuous) complex functions on Delta to an operator algebra A with their self-adjoint operators usually 
associated with observables of a quantum system. In [4], corresponding models, here referred to as NC-OOMs, 
have been introduced and investigated in detail for a class of stationary states on quasi-local C*-algebra that 
feature finite process dimension as introduced in Definition 13 below. In what follows we do not impose this 
restriction. 

Let .4 be a finite- dimensional C*-algebra with unit 1^ and positive cone A+ = {a G A \ a > 0}. With 
A^ := A®^, we denote the C*-algebraic tensor product (i.e. the norm completion of the algebraic tensor 
product), and similarly, Ai := A^^ for 7 C Z. Let S{A) denote the set of states on A, i.e. <S(^) = { p G 
A* I p positive, p{tA) = 1 }; where A* denotes the dual space of A. Note that if ^ = C(A) = C^, then 
Am = C{A^) and SiAn) can be identified with P{A^). 

Definition 10. (y,T,v,i) is an NC-OOM with output algebra ^ if V is a vector space, T: AxV ^V, 

(a, w) Ta{w) is bihnear, v €V and £ € V* such that for n S IN, ai, . . . , a„ G A+ 

1. £{v) = 1, 2. ioTt^ = e, 3. eoTa„o---oTa^{v) > 0. 

The state f on A]^ obtained by linear extension of 'fi{ai (8> ■ • • (8> fln) ■= i° Ta„ o ■ • • o [v) is called generated 
by the NC-OOM. 

Remark, a) The state 9 generated by an NC-OOM is a well defined state on An- Note that it is not 
necessarily translation invariant. 

b) We adapted the definition of finitely correlated states given in [4], to fit the classical OOM definition, see 
Definition 1. In [4], the Ta^^ are applied in reverse order and t need not be normalised: there ^p{a\ (8) • • • (g) 
•^i) ^ T{e) ' ^ ° -^ai ° ■ ■ ■ ° Ta„ {v) ■ Notc that as a consequence of the reverse order combined with condition 
2. the associated finitely correlated states in [4] are translation invariant by construction: 

<^(1^ <8) ai O • • ■ a„) = Ti^ o o • • ■ o T„„(w) 

Let A^c(A'^) be the set of complex-valued measures of bounded variation on A'^, and i): Mc{A^) 
C{IS^)* the natural isomorphism. Then every probability measure P G 7^(A''^) corresponds to the state ^^{P) 
on the commutative C*-algebra C(A'^). In the same vein, OOMs with output alphabet A can be interpreted as 
the special case of NC-OOMs with commutative output algebra A = C(A). More precisely, there is a natural 
one-to-one correspondence l as follows. If O = {V, {Tci)d,eA,v,£) is an OOM, the corresponding NC-OOM is 
l{0) = {V,T,v,£), where V = V ®iV is the complexification of V, and £ is the complex-linear extension 
of £toV. T is given by T(/, w) := EdeA f{d)Td{w) iov f e A = C(A), w &V, and extended linearly to 
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w G y. Obviously, dimE(y) = dimc;(V). Furthermore, it is straight-forward to check that if O generates 
P G P{A^), denoted by gen(O) = P, then i{0) generates V(-P) G S{Ak), denoted by gen(i(C>)) = tpiP). 
This means that the following diagram commutes: 

OOMs(A)^^ NC-OOMs(C(A)) 

gen 

p(AiN)C S{C{A^)) 

The canonical NC-OOM of a state ip G 5(^]n) is defined similarly to the canonical OOM of a classical 
probability distribution, cf. Definition 2. In more detail, the dual of corresponds to the space Ai{A^) 
of signed measures used in the classical construction. The initial vector is (p itself, and the evaluation 
functional is the evaluation at i-^- ^-a{p) = pi^An)- The map t-^: Ax A^^ — >■ A^ is defined by 

T-^{a,p) := T^{p) := p(a (g) • ) := {X ^ p{a (g) X)) , 

and again we set 



7 — 1= T 



(1) 



Definition 11. For ip e <S(.4]n) let 

:= span{ r^. ..„„(¥') | G INo, ai, . . . ,a„ G ^} C ^J;^ 

and denote the function A x ^ V^, {a, p) i-^ Taip) '^itih a slight abuse of notation again by r-^. Then 
{V^,T'^,ipjA) is called canonical NC-OOM of ip. 

Remark, a) is a vector space and t^, for all a G A, maps into V^. 

b) The discussion in [4] is about translation invariant states on Az- There, the image W^f, of the map 
A-n^ A"^, a i-> <^(a (8> • ) is used instead of V^. In general, we have the relation C C T^*. In 
the finite-dimensional case, however, the two spaces coincide. 

c) Let P G ViA^), and O the canonical OOM of P. Then the corresponding NC-OOM l{0) is the canonical 
NC-OOM of tp{P) up to the identification of Mc{A^) with C(A")* by the isomorphism ip. In particular, 
Ta = EdeA ■ Ip o o ip-'^ and 

dim(P) = dim(V'(P)). 

d) Note that is weak-* continuous. 

Lemma 12. The canonical NC-OOM of (p G S{Am) generates ip. 
Proof. Let ai , . . . , a„ G A. We obtain 

^AOT^,...aA^) = ^al(T-a1...a„_iH)(l^H) = ^^...a„_i MK) = ••• = ^(tti ® • • • (g) a„). □ 

Similarly to the definition of process dimension of a probability distribution as given in Definition 4, we 

propose: 

Definition 13. The process dimension of G 5(^]n) is the dimension of its canonical NC-OOM: 

dim((p) := dim(F^) G ]NU{oo}. 

5 Properties of Process Dimension 

In this section we present our main results: lower semi-continuity and an ergodic decomposition formula for 
process dimension. In the classical special case, corresponding results for a class of complexity measures have 
been obtained in [9, 8]. For the technical prerequisits that are required for our non-commutative extension 
we refer to the books [1, 10]. 

Theorem 14. The process dimension dim: >S(,4]n) — > INU {oo} is weak-* lower semi- continuous. 
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Proof. Because Atm is separable, iS(^]n) is weak-* metrisable and thus sequential semi-continuity implies 
semi-continuity. Let (p be the weak-* limit of a sequence {y:>n)neK iii '5(^]n) and dim((^) > d. We have to 
show that dim((/3„) > d for sufficiently large n. Let (V^,t"^,<^, be the canonical NC-OOM of <f. Since, 

by Definition 11, dim(y^5) = dim(<y9), we can choose linearly independent vi,...,Vci € V^- Moreover, by 
definition of V^,, there exist Okj E A, 1 < k < d, 1 < j < nik, such that Vk = t^^^ i'P)- For n e IN, 

we define vectors := t^^^ (ipn) in respectively. Due to continuity of t^, we have v]!: Vk- If 

v", . . . , are linearly independent for all sufficiently large n, the proof is finished. Suppose this is not the 
case and w.l.o.g. that they are dependent for all n. Then there are A" £ [^1:1] with maxk |A^| = 1 and 
Sfc=i '^fe^fe = for all n. Because [—1, 1]'' is compact, we may assume by passing to a subsequence that 
"j±|P fgj. some Afc. Due to weak-* continuity of addition and scalar multiplication, J^k^kVk = and 
hence Afe = for all k, in contradiction to max/; |A^| = 1. □ 

Due to the one-to-one correspondence between canonical OOMs of classical processes and canonical NC- 
OOMs of associated states on (abelian) C*-algebras the above theorem has the following corollary. 

CoroUciry 15. The classical process dimension dim: V{A^) ^ W U { oo } is weak-* lower semi-continuous. 

We now derive an ergodic decomposition formula for process dimension in the stationary case. Let 5s (^w) 
be the convex set of translation invariant states, and 5e(.A]N) C <Ss(»4]n) the set of ergodic states, i.e. extreme 
points in <Ss(v4]n). Since Ajt^ is asymptotically abelian (w.r.t. the shift) and >Ss(^]n) is metrisable, <Ss(v4]n) 
is a simplex and the set 5o(^]n) of ergodic states is measurable in 5s(w4k). In particular, every translation 
invariant state ip has a unique ergodic decomposition G V(^Ss{Am)), which is supported by the ergodic 
states, ^{SeiAn)) = 1, and if is the barycentre of v, 

(fi = j id dv. 

Moreover, in what follows, we make use of the important fact that in our situation the ergodic decomposition 
is orthogonal. For details, see [1, Sec. 4.1, 4.3.1]. We obtain the following ergodic decomposition formula for 
process dimension. 

Theorem 16. Let G »Ss(.A]n) be a translation invariant state with ergodic decomposition v G V{Sf.{AwS)- 
Then 

dim((p) = ^ dim(^). 

V'Ssupp(i/) 

For the proof, we use the following two lemmas. 

Lemma 17 (representation on Hilbert space). Let ^ G <Ss(.A]n) and {'H(p,Trif,,^,p) be the GNS-representation 
of Ak w.r.t. if. Then there is a linear injection from into with 

p{X) = (7r^(X)t^(p), ^^)^ VX GAK,pe 

Proof. Let u be the unitary representation of the shift on An- Then stationarity of ip implies u^^ = For 
ai, . . . ,an & A and A = ai (g) • • • (g) a„ let 

and extend the definition to ^4 G linearly. Then we have for A = an 'S' ■ ■ ■ <S> Om 

T^ip)iX) := ^r,t...,,J(p)(X) = {{un*n^{X)u-n^{A)^^, i^) ^ ^"«-= {^AX)U, C^)^, (2) 

where t^j^...^^.^ is defined in (1). For every p G V^, there is an n G M and A G A<^oo '■= UnsiN - 
with p = T'^{p). Define i^p{p) := £,a- Because is cyclic and .4,<oo is dense in .Am, (tt,^{X)C,\, i(p)^ = 
(tt^{X)C,2, ^v)^ for all X G A<oo implies that d = ^2- Thus t,^ is well-defined. Injcctivity is obvious, 
because p can be recovered from by (2). □ 

Lemma 18. Let y G <Ss(.4.]n) and ip = J^ipe^ ^(V')V') where ^ C <Ss(.4]n) is countable and ^{i)) > 0. Then 
where is defined in Definition 11. 
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Proof. "C": By linearity of t^, we obviously have C V^. 

'O"; Because i'{tp)ijj < f, there is a ^i/, € Hep with u^^ = and ip{X) = (7r^(X)^^, ^,^)^. As is cyclic 
and »4<oo is dense in Afi, there is a sequence An € .4.{i, ...,„} with 7r,^(A„)^^ — >• Let p„ := t^^(i^). Then 
Pn e K^, and for all X G 

|H| = ||7r^(Jf)||<oo 

Thus Pn-^ ip and e V^*. As is weak-* continuous, t^(V^*) C V^*"*, hence 1/0 C V^'"*. □ 

The lemma shows that in order to represent in terms of NC-OOMs, we have to represent all ergodic 
components tp of (p (and not more). We still have to show that they have to be represented independently 
without synergies. This follows easily from the orthogonality of the ergodic decomposition. 

Proof of Theorem 16. 1. Finitely many ergodic components: In this case, Lemma 18 directly implies "<." 
For ">," we may assume that dim(F^) < oo and thus also (im\{V^) < oo for all ergodic components ip. 
In particular, V^p = V,p by Lemma 18. We can identify the GNS-Hilbert space with a subspace 
of Hep and because the ergodic decomposition is orthogonal, the are mutually orthogonal. Since 
i-viVil)) ^ H^, by Lemma 17, the sum of vector spaces is direct, i.e. = 0^V^, and dim(V^) = 
X:^dim(VV). 

2. Infinitely many ergodic components: The sum on the right-hand side is infinite. To see that also 
dim((p) = cxD, fix n G M and choose a decomposition of supp(i/) into disjoint measurable subsets ^'/j, 
k = 1, . . . ,n with positive z/- measure. Define Vfe := J^^, iddu to be the barycentre of z^t^j.- Then the 
decomposition tjj = tl)^ is orthogonal and by the above argument dim(y^) = dim{V^^) > n. □ 

Note that Theorem 16 provides an alternative proof of Corollary 8. 
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